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L A meibod of encoding a video signal comprising the steps ol: 

leoeivtng a digital video signal including a succession of digital 
representaUons related to picture elements 6[a video image where said digital video signal 
5 has a characteristic resolution; 

producing a first encoded version of said received digital video signal 
having a resolution less than or equal to said characteristic resolution; 

produdng a second encoded version of said recdved digital video signal 
having a resolution equal to said characteristic resohiticHi; 
10 producing a first prediction of said video inu^e from said first encoded 

vcRion of said received digital video signal; 

producing a second prediction of said video image from said second 
encoded version of said received digital video signal; 

combining said first prediction and said second prediction to produce a 
IS combined predictioii; and 

emplo^f 'mg said combined prediction to encode said second.encoded.version 
of said received digital* video signal; 

2* The method of claim 1 in which said step of producing said first 
piedictioo Indudes pcoducmg a spatial prediction. 

20 3. The method ofdatm 2 In which said step ofprodudng said second 

prediction includes producing a ternporal prediction. 

4* Themethodof claim 3 in which said step of combining includes 
weighting said spatial prediction and weighdng said temporal prediction to produce said 
corribined pfecfiction. 

25 5. Themethodof claim 1 in which said step of encodmg a first encoded 

versiraiitncludes encpdii^ an interlaced version of said received digital video signal. 

6. The method of claim 1 in whidi said step of encocfing a first encoded 
version includes encoding a piogiesave vcr»on of said recdved digital video sign^ 
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7. The method of claim 5 or cimm 6 in which said step of encoding a second 
encoded vetsbn includes encoding an interlaced version of said received digital signal. 

8. The method of claim 5 or claim 6 in which said step of encoding a second 
encoded version includes encodir^ a {Kogressive version of said leceived digital signal. 

9. The method of claim I in which said step of encoding a first encoded 
version includes encoding using MPEG-1 coding standards. 

10. The method of claim 1 in which said step ofcncoding a first encoded 
version includes encoding using H.261 coding standards. 

11. The method of claim I in which said step of encoding a ffarst encoded 
version includes moding using MPEC-l cocting standards. 

12. The method of claim 9 or claim iOor claim II in which said step of 
encoding a second encoded version includes encoding using MPEG-2 encoding standards. 

13. A method of decoding said first encoded version and said second 
encoded version of clum 1 fcHT producmg an unencodedvidea signal having at least one of 

15 a plurality of predetermined charaderisdcs. 

14. The method of claim 13 wherein said at least one of a plurality of 
predetermined characteristics is resolution scale. 

15. The meOKxl of claim 1 3 wherein said at least one of a plurality of 
predetermined characteristics is pictiire forinaL 

^ 16. An apparatus fcH' encoding a video signal comprisirig: 

a receiver for receiving a digital video signal inchidmg a succession of 
digital representations related to (touie elements of a video image where said digital video 
signal has a characteristic sesoludon; 
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a means for producing a first encoded version of said received digital video 
signal having a resolution less than or equal to said characteristic resolution; 

a means for producing a second encoded version of said leceived digital 
video signal having a resolution equal to said characteristic resolution; 

a means for producing a first predicii wi of said video image form said fust 
encoded version of said received digital video signal; 

a means fcM-producmg a second prediction of said video nnage forms said 
second encoded version of said received digital video signal; 

a means for combining said first piedictiOD and said second prediction to 
produce a combined prediction; and 

a means for empby ing said combined prediction to encode said second 
encoded version €i sud received digital video signal. 

17. The apparams of claim 16 in which said means for producii^ said first 
prediction includes a means for producing a spatial prediction. 

18. The apparatus of claim 17 in which said means for producing said 
second prediction inchides a means for pnxittdng a temporal prediction. 



19. The apparatus of claim 1 8 in which said means for combining includes 
a means for weighting said ^mtial predictioD and a means for weighting said temporal 
prediction to {HDduce said combined prediction. 

20. The apparatus of claim 16 in which said means for encodmg a first 
encoded version includes a means for micoding an interlaced version of said received 
digital video signal. 

21. The apparams of claim 16 in which said, means for encoding a first 
encoded veisioa includes a noeans for encoding a progressive version of said received 
digital video agnal. 



22. The apparatus of claim 20 in which said means for encoding a second 
encoded veisioQ mchides a means for encoding an interlaced versioi of said reoehred 
digital signal 
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23. The apparatus of claim 21 In which said means fc)r encoding a second 
encoded versiott inchides a means for encoding an interlaced version of said received 
digital signaL 

24. The apparatus of claim 20 in which said means for encoding a second 
encoded version includes a means for encoding a progressive version of said received 
digital signal. 



25. The apparatus of claim 21 in which said means for encoding a second 
encoded version includes a means for encoding a progressive versicm of said teceived 



26. The apparamsorclaim 16 in which s^ means forencodmg a first 
encoded version includes a means for encodmg using MTCG^I codmg standaids* 

27. The apparatus of claim 16 in which said means for encoding a first 
encoded veisicm includes encoding using H.261 coding standards. 

>5 28. The q)paratus of claim 1 6 in which said means for encoding a first 

encoded version includes a means for encoding using MPEG-2 coding standards. 

29. The apparatus ofclahn 26 in which said means for encoding a second 
encoded version mchides a means for enoodb)g using MPGG-2 encoding standards. 



30. The apparatus exclaim 27 in which sud means for encoding a second 
encoded version inchxles a means for encodmg using MFEG-2 encoding standaids. 



31. The apparatus of claim 28 in which said means for enoodiiig a second 
2S encoded version incudes a means for encodii^ using MPEG-2 encodbig standards. 



SPATIALLY SCALABLE VIDEO ENCODING AND DECODING 
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Tills invemion relates to encoding and decoding of video signals and. more 
particulariy, to efficiem encoding of video signals in a scalable mamier which pennits 
5 ^<*«>>ntt««tobcdccodedina variety of resohitionscalcsa^ 



Worldwide efforts are underway to improve the quali^ of video signal production, 
transmission, and reproduction because a great deal of commeicial unpoctance is being 
predicted for improved quality video systems. Ihtse efforts involve, at least in pait 
increasing the resolution with which images are converted into lepresentativc etoctrical 
signals, typically in the form of digital bit-streams, by increasing the spatial and temporal 
samplmg rates that are used to convert video images into electrical signals. This increase m 
resohition coosequcmly meaiB that more data a^^^ 
and transmitted in a given time interval. 

Video images, such as those images in the field of a television camera, are scanned 
ot a predetermined rate and converted into a series of electrical signals, each electrical 
signal itpiesenting a diaracteristic (rfa pw^ 

as a picture element, pel, or pixel. Picture elements are typicaUy giooped mto mactoblocks 
for most video signal processing purposes ^itiore each macros 16 by 16 

array ai picture elen^ts. A frfurality of macroModcs taken togeto at a predetermined 
instant in liine fonns «^ anwunis to a sti^ 

of the image at the predetermined instant of time. Increasing the quality of video signals 
produced in this mamier involves, at least in part, the use of larger number of smaller-size 
picture elemems to represem a given image frame and the productiwi of a larger number of 
image frames per unit time. 

As &e number of pels for each video image increases and tte rate at which im^es 
are produced increases^ there is an iiKaeasii« amoum of vi<teo data which must produced, 
transmitted, recdved and processed in a given time interval A number of video 
conqwessiOT schemes have been proposed which attempt to transmit higher quali^ video 
images ushig the sane number of hits and the same bit rates used for lower quaUty images. 
Tht Motion Pictures Expert Group Phase 1 (MPEG-1) standard provides a particuter 
syntte and decoding process for one such sdieme. Tliis standard is set f 
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Sla«la«U OigaaiMk. (ISO) C<^^ 

Assoc««dA«liolbrI)i8itolSlon«eMedl8«ttptol^Mbit^s."N<^^ 
Itnuvbedeslrate 

oansmoted higlM«ol„ao„ vi<tea signal. Fbr example, a video signal dm»lu«co«sly 
«~kied to both higlwleflnitk» television imXTV) and standani msc tefevisior, 
i««*m may havelo provide lmagetlunringave,yWghdegiee of resolution 
leceivws and images having a lesser degiee of iesoluik» to the standaid leceiven. 

Suwlariy, ihe degree of image resolution which needs to be obtained &om a video sig^ 
displayed oa a windowed oom|»ler soeen must by varied witfi die sl« of the paiticuter 
wfadcm in which It is displayed Other application m 

are desirable include video confetenctng where different video eqnipniem may be 
employed at each locatioo and video tnmsmilted over asynd^ 



One known method of providtng a video signal 6om which onaBcs of vaiyii« 
n»ohaioo may be derived is to simnltaneottsly inmsmit a ret of fadepead^ 
video sequence, each replica being scaled for reproduction at a difftoent level of resolution. 

TWs appw«lv Imown as 'ymnkawing,- is simple, bw it requires ina^ 
a«»mmodate the tmnsmissioo of mnltipte h«lepende« video hnages. A 
efficient alternative to simulcasting b scalable video. Resolutton scataUe video is a 
technique in which a video signal is coded and die resulting bit^sequence is partitioned so 

that a nmge of resohitlon leveb mv by derived fiom it depending upon the paiti^ 
signal deoodfaif Khenw empk^ at die receiver. 

RescAitioo scabbte video codmg may be adaeyed eittw in the spatial or fiequency 
domra^^patial scaiaKUty uses toyend coding, typically inchidmg a base-byer and an 
eahancenHnt-lqiar In die sfa^ (kxnain. where dwie is a loose coupling between the 
hgws, that is». die coding algocidims used to code die layers are mdqieadeitt; but the 
enhanccanert layer is coded ustag die recoDstni^ 

coding scheme.used for die two layers can also be chosen mdependeotly. as can the 
paiticniar medipds of 1^ wl down sanqpling. 

Untetunaidy. die codfaig of resotetion scatable vfafeo is not provided widiin die 
constraints of most video stmidards. A particular limitation of die MPGG-I codfog si^ 
is its lade of pravisioos facilitating resolution scalable video encodii« and H^n^^tg 
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Video inu^ges of vaiying rtsolutkro arc derived IhTO 
bandwidth efficiency, in accortance wiib ibe principles of the Invention, by employing a 
new t«oJ«yer video coding technique using spatial scalability in whicte the prediction 

taken from one layer is confined with d» prediction tdcen fhHn the oa«r 1^^^ 
combined predictioo is used to code one of the layere. 

^ " iUustrative example of the invention employing a base-layer and an 
enhanoementJjqwr, the spatially interpolated base-layer is combmed. by the seieciion of 
appropnate weights, with the motion compensated innpocal p,«iiction of the 
enhan«ment-layer to generate the prediction used to emxxle the enhancement-hyer. 
Weigto ate selected based on a calculation of the sum of the absolute diffeieoce^ 
smn of the spates of the differences between the pn«lictioo a^ 
produce the prediction givmg the best bndwidth efUdeney. Hiis weighting process is 
called spatio-temporal weighting. 

Odw aspects of iUustrative examples of the invention include using spatio- 
tempowl weightfaig where die base-layer and enhancement-layer have panicubv picture 
fomiats that may be reqnimi in certain applicaliona of the faivaidon. R>r example, in one 
application the base-toyer may need to be in an interiaccd format, while in anodier a 
pfogiessive format is required. Ilius. there iire four iUustrative forms of s|»tial scalabiUty 
employfag aspects of the iBveotfcm that lesBlt fami the various combinato 
enhancement-l^ pictnie fbnnatt. These ate: progrwalve^oiwigressive. piogrwslve^o- 
ii»leriace, interiace-toiKogressive. aid interlace-to -interlace. 

The invention provides substantial in^rovements over prior art techniques of 
lesohitioo sealahfe videti. Rirexaii^il^ bSBdwidUi d 

of weights can be selected for eat* form of spatial scalaWll^r. the howm ^ 

provide forconqKOibiUQr between a number of diffeient codmg standards; and the layers 

mqr be readOy prkritiwd for tiansntissioii <m networfts osmg multipte priori 
robust and error-iesiUenttransmisstoa 

Tiie discusaiai in this Summary and the foUowiitg Brief Description of the 
Drawing. DetailBd Descrqilion, and drawings merely rq^esents exain^ 
and is opt to be coiuidered in any way a Umitation on tte scope of the exclusionary rights 
coiifcnBd by a patrat whidi may issue fiom this lyplicatkm. The scope of such 
exdnsidnaiy rights is set fiorib in the darnis at the end of this ^qdlcation. 
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Ha 1 shows, in simpUfied block diagram fomu an illustrative two-liQ^ encoder 
and decoder embodying aspects of the invention. 

no. 2 shows a block diagram of the decimadon operation (DEQ used in the 
progressive-toiwogrtssive and progiessive-to-interlace forms (^spatial scalalniity. 

RG. 3 shows a block diagram of the inteipolatfon operation QHTP) used mthe 
iwogressive-ioijrogressive and progressivc-to-interlace fonns of spatial scalability. 

FIG. 4 shows a Mock diagram of the decimation operation (DEC) used in the 
Mrtcrlace-to-progpessive form of ^)atial scalability. 

RG. 5 shows a block diagram of the interpoladon opcfatioo (INTP) used m die 
interlace-toiMOgxessive fom of ^»dal scalability. 

FIG. 6 shows a block diagiaiii of die decimation operation (DEQ used in d» 
interiace-lo^nterlaoe fom) of spatial scalability. 

HG. 7 shows a block diagram of the interpotetion openoion (INTIO used m die 
intertace-to-mterlace form of spatial scalability. 

FIG. 8 ahowsdiedctaibof dieinterlacc^o-piogiessive intBip<^^ 
in interlace-to-interlace form of ^>atial scalabili^. 

RG. 9 shows die detaib of the intBriacc4«>^>rogi^ 
in die interlaoe-tD-piogiessive form of qwtial scalalnUty. 

RG. lOshowsdieivinciplesbehindtfieweigfaiedspatio^ 
piPgrcssive-toiwogicssive wl interlace-to-^ogiessive forms of spedal scalabOi^ in 
aocxndance widi an aspect of the inventjkm. 

HG. 1 1 shows d» principles behind dw wdghted spatio-iemporal prediction for d^ 
pn«icssive-lD-imerlace and Interlace 
25 with an aq)ect of the invention. 

FIG, 12 shows a diagram of die base^layer encoder and an enhoicement-layer 
endxler used la die Illustrative embodin»M 

FIG. 13 shows a two-layer decoder c o nesp^ Hi d hi g to ilhistrttive embodiment of 

RG.i. 

30 .PIG. 14dio«»a«nq)lifiedbkji*diagramofa^^ 

aspects of die inveatioQ. 

- FIG.15showsashiiplifiedbldckdiagramofaspado-teni^^ 
aspects of die mvendoa. 

The following abbieviadotts have been used in die drawings listed above: 




BF-huffer 

GOMP-conqiarator 

DEC -decimation 

INTP . interpolation 

MC - motion compensation 

ME - motion estimation 

mv- motion vectors 

ORG- organizer 

Q - quantizer 

IQ - invme quantizer 

QA- quantizer adapter 

T- transform (e.g.. a Dbcrete Cddne Transform (DCTJ) 

IT- tnyeise transform 

VFB - variable and fixed length encoder 

VFD - variable and fixed length decoder 

STA - spatio-temporal analyzer 

PS - previous picture store 

NS - next picture store 

SW- switch 

WT-weighter 



FIG. I shows, in simpfiiied block diagram form, an iOostiative two-layer encoder 
and decoder, embodying aq)ects of the invention, inchiding cnhanccmcnt laycr encoder 
1 180, bas^]ay€T encoder 1 140, enhancement-layer decoder 1340, base-layer decoder 1300, 
and othor elemnits. Enhancemem-li^ encoder 1180, base-layer encoder 1140, 
cnhmccment.4aycr decoder 1340, and base4ayer decoder 1300, and die functions 
contained therein are (tesoibed in detail below. High resolution video sigi^ VIDIN enters 
on input Ime 1 100 and pa8« to spatial toamaiw 1 120 on line 1 ^ 
vidteo signal VIDIN may be tow^nss filtered before spaM decimate 1120 reduces ibc 
number of pichire eloneots to a lower resolution called the base-layer lescrfnticm. High 
resoltttiin video signal VIDIN may be fcmnatted as eidier interlaced or progressive. It will 
be ^iprsctated by those skilled in the ait diat it may bo desirable, in some exan^ of the 
mvoitioo, to use tenei>icaire coding for iniertaoed vkleo ^nals in accordance widi die 
Motion Pkmne Experts Groi^ Phase 2 Test Mocfei 5 Draft Versiott 2, Doc. MPEG93/225, 
April 1 993 CMreG-2). Alieniativdy, it may be cfesiiable to en^by field-picture codmg in 
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«^deam«.on(DEQ.Al«K«ghdeci™»«„a,eweHh^ 

5 encoder 1140. whicboiitpotseicodedbft-sliwmiBLoolTO 

line nSl'''!;!^''*'''"*^'""'"^^*"^'""-'*^ 

l«e 1 150 to spadal ,««poIalor 1 160. Spatial imeipobtor 1 1 60 ina«« the number of 

pd. tame «sl^ . method of „p««py^ in^potatioB ONTP). Although interpolator 

wdl 1^ u. .he .pedfic me«»d, of «p««vling hMev^ 
aspects of the mvention. are discussed in detail below. TT« upsamp^ 
signal is output on line 1170 to enhancement-layer encoder 1180 which outpau encoded 
bjl^ELon B«e 1200. E«h««me.,,.ta^ 
fio«lnKll70a8ap«ilc,io,,mori«toa«l«^^ 
the high resohition video signal input on input Hnc 100. 

Encoded bit-stieann BL and EL at die outpw of base-layer encoder 1140 and 
enhancemem-teyer encoder 1180 a« combined m multiptexer 1250 in ptepamion for 

tnasmission on cham^l 1 260. Alteinath^ly. bit-st«m» BL and EL could be sent OD tw 
seprnue and independem channels. The encodmg of higb-iesohition video signal VIDIN 
into bit-sneams BL a«l EL advantageously allows the use of prioritizatioo for tnmsmission 
on netwoiks using multiple priorities which focilitaies more robust and enor-nnilient 
transmissioa. 

If bitHmearos BL and EL are multiplexed, then, after tnmsraission on channel 126a 
<lemnltiplexer 1270 sepante bit-stieains BLand EL and outputs Wt-stiw^ 
IhMs I290and 1280,a8suimngdieiearei»tnmsmissionein)isonch«ncl 1260. 

Bit-sticams BL and EL are input into base-layer decoder 1300 and enbancemem- 

teyer decoder 1340on Imes 1290 and 1280. tespectively. Base-layer decoder 130O outputs 

a decoded base-bycr video sigmd VIDOinB «» Hne 1310, which, in the absence of 

•nnismiisioii eiiOB. is exactly the same as the leplica decoded vidw 

I>«oded b«»^jw video signal ViroirrB is also m|Wt ol^l^^e 1315 to 8^ 
inleipblator 1320. which is a dnplicate of mteipolator 1160 and which pnxhices m 

upsampled agnal oo line 1330. In the absence of tiansmission eiiws. the upsamp 

signal on libes 1330 and 1170 an identical fiihattcemeiit.layer decoJer 1340 utilizes the 

iqpsanvied video on line 1330 in coojunelioD widi the ci^^ 

1290 to pn)dnceadccodedhigherres<rfution video signal VIDOUTBon output Itoe 1350. 

As will be appeciaied by ttiose dulled m the ait Uttt it may be desirable, 
ap^icatioBs, tn high resolntiin video n^ut qgnal VIDIN at m^ Une 1100 to be of 



progressive fonnat. what as other apirfio^ 

mteriaced fotmaL Thus, four forms of spatial scalabUities may be used tt> iOustiflle the 
prmdptes of the invention that depend on the base-layer-to-enhancemeat-kver picinn 
fonnats. Hie illustmive fomis of spatial scaiabOity ate called propessive-to-pfogiessive, 
piDgressive.tt>.interlaoe. inierlace^o-ptogressive and interiace-lo-interlBce. Depending on 
die form of spatial scafadriliiy. ibe spatial decimation (DEQ and spatial inieipolation 
(INTT) operations discussed above may be different n>e DEC and WTP operations 
necessary for each of die four aforementioned forms (tfqpatial-scalability and fhe potential 
application of each are discussed below. 

FIGS. 2 and 3 show tiie DEC and inteipoladon INTP operations required for tf» 
ilhistrative ptogressivMo-progressive and progressive^o-in^eriace fcnns of spatial 
scalability. As shown in HG. 2, in ti« DEC operation, high resolution vidM> signal in 

piijgressive fcnnat is ii^pui on Ime 2110 to horizontal and vertical decimatoe 21 15. where 
high resolution video signal SldNp may be filtered before borinntal ami Uitlcal 

decinwoi 2n5 reduces die number of picture elements contained in high resolution video 
signal SIGINo in bodi die horizontal and vertical directions by reducing the rate at which 
high resohitioo video signal SIGINd » ampled. The lower spatial resolution video 
SIGOUTd » output « Une 2130. In some applicationt it may be desir^ that no 
reduction in sampUng rate bdow 1:1 occur m other die horizontal or die vertical 

d&ections. TTie rae « which horizontal and vertical sampler 2115 samples hi^ resolution 
vid» signal SIOWo; is wqwBssed as a ratio between two integers, for insu^ 
hofizootal decimation requires reduction of die samplii^ rate by a foctor of 2 horizontally. 

As shown in HG. 3. in die INTP operation, low resohition video signal SIGIN, in 
piQgressive format enters on line 3150 to horizontal and vertical iaieipolator 3155 which 
pcrfwiBstfieinvene operation of dMttpc rfo r iu e d by horizontal ami vartitai^ 
fflO. 2). However, since a loss of spatial lesohitim in dednutor borizoatal and vertical 
«iedinator 2115 ocars, horizoittal and veitical inteqioiauir 3155 can only ouqwt an 

qiproximatioo (tf die signal at die input to dedmalor 2110 (HG. 2) as SlGCnn; o^ 
3170. 

' An exampte of die pragressive-to-pngressive fonn of ^adal scslabiUiy b a Go^n^ 
scheme where a Common biterraediatB Fonnar (OF) type signal is inpot to base-layer 
encoder 1140 (FIG. f) at a resolution of 352 horizontal x 288 vertiod pcture elements per 
pn^gressive fiame. and. a Siqier Ctunmon btennedialB Faimat (SdF) type signal is input 
toeahancement-l^encoderllSOCnc. I) at a resohition irf 704 hotizonialxS76 vertical 
picmre elements per progressive ftaroes. where bodi iiqnt signals Inving a ftame rate of 30 
(raines/sec. OF and SCIF fannaded signals ae weU known in die ait The CIF signal inptit 



to b»eJayerenoote 1140 (Ha 1) I, derived fton. tte SOF dgnd 

us«« a facta of 2 in both the horizontal and in veitical dii«^ 

layer lhm« « Inwpoltted ly a filter of 2 both horizontany and 

m the prediction u»d to encode the signal in the enhanceiiwt-toyer. Allbotigh tUs 

5 wpi'^ a factor of 2 for horiromal«Kl vertical^ 

oihttimeger ratios can also be advantageously used, as wiU be appreciated!^ 
m the «. to this eKanvte. a Motion Kctuies Expert Group Phase 1 (KII«^ 
Itawnmeodation H^l-Video Cbdec for Audiovisnal Services at p»64 Knil/s. Geneva, 
Angust. 1990 (H^I) coding scheme k used to code the hase-kyer to IBustrai^ 

10 scalability, in accordance with an aspect of the invention, can advantageously permit 
co«patibi% hetw«n the MPBO-I. or H^l. and st^^ 
and may be desmUe in some applicatioM to nse MPE0.2 coding in both layeis. Ftor 
example, the base-byer may eni|4oy MPEG-2 mafai^le codmg and the eiyiancemeiit- 
layer may employ MPEO-2 nexi^iiofile spatially scatable coding. Both main^irofile and 

IS next-pnOeoodfaigschemea are known in the ait 

An example of progiessive^o^nteilace scalabOity fa a codmg scheme where 
Source Inpw I^rnw (SIF) type signal is ii^t to base-toyer encoder 1140 (Ho! I) at a 
lesolution of 352 horizontal X 240 vertical pictures elements per noninteri^ 
ComHe Cbnsultatif buematiclial des Radiocommnnications Recommendation 601 
» Standard 4:2K> (OCIR-601 4:2:0) type signalla input io eahancohent^ encoder 1180 

(RG. 1) « a resolutioo of 704 hotimotal X 480 vertical picdire elements per interlaced 
ftame. TTiB SIP signal inpm to tese-kyer encoder 1 140 
«l *2« signal by dropping the ev«i4leldsi follows 

in horizontal directioit Ixcally decoded hase-teyer ftwnet ate upeanpled by a Ik^ 
2$ horaomaliy and vHtically and ised for fte prediction used to enco* 

1^. In this ojfflnple. ffl MITO-I coding sdwme is used to encode die base^^^^ 
die above exanyle. to iUntintB how spatial acalM^ 

Invention, em advaMe^eously permit compuibiU^ between die MPEO-1 and MPGG-2 
staiulioJi. It is also possible, and may be desirable in some applications to use MreG-2 
30 coding in bMh layqfs. For exan^ the base-layer nu^ enq>loy MI^2 main-piofile 
coiBng and die en han cem emr laycr taay employ MPBG-2 next-profile qmtially scalable 
coding. 

RGS. 4 and 5 show the DEC and INTP operations rettuiied for the iUuslrative 
interiace^o^xogicssive fran of qndal scdalnliQr. As shown in HG. 4. in die DEC 
35 opeittioiu .higiitesofatiim^^dm signal SIGINpj is input ooUne 41 10 to hori^ 
y^ftkal dedmator 41 15 «4itdi itedum die mm^ of pi^^ 



10 



vertical di«ctk« IV «dudngtte 

Alower spatial ««,Itttion p™g««ivc video signal i» ao^»t on line 4415. In some 
»PI*cations.itn«vbe<le8lraWetta 
the lK*«ontal or the vertical directions. TT» sami^ng 

deamaior 41 15 is expiessed as a lado between two integeis. for instance 2:1. Horizontal 

*ciniatioo requins •eduction of sampling rate by a factor of 2 horizontally. Next 

piogressive ftames at line 4105 further undogo a prog«8sive^«>™terlace dec>n>atio.i 

opoBtlon that U weU known m the art. Lower spatial msolution into 
output on line 4l30as SfOOUToj. 

As shown in ITO. 5 Jn the Ihm* operation, low ,e«rftttion interlaced video sig^ 
SIGINn enters on Une 5150 to interlace^o^«)gressive inteipolalor 5165. Ftogtessive 
lower lesolution frames on Kne 5145 aie output of intertaewoiwogressive imerpolator 
5165 and aie Ibd as an inp« to horizontal and vertical interpolator 5155 which perfofn» 
tt» mveise openition of that perfomied in horizontal and vertical dedmator 41 15 (HG. 4). 
IS However, since a toss of spatial resohition in horizontal and vertical decimator 4115 

occurs, horizontal and vertical faMetpolalor 5155 can only output an approximatipn ^ 
signal inpot on line 41 10 (FIG. 4) lo horizcmtal and vertical decimator 41 1 5 (HG 4) on 
line 5170 as SIGOUTq. 

An example of ioteriace^ptogtesstve scalability is a scalable coding scheme 

0 where the base;-bycr emptoys die main-profile coding of CCIR.601 4:2.-0 lesohition 
inleriaced lianies and the caliancenicm-bq«r employs progiessWe hi^ defmitian television 
(HDTV) resohition frames at 60 fianesfeec. GClR-601 *2.-0 mahnxofile and HOTV 
formatted rignab are wen known in die ait Such iniertac©^oi«)gies8i^ 

can be advantageously used to achieve digital piogiessive-HDrV compatibility with 
s ««w»aii digital TV. h Ibis ejtanqte.MI^ 

the base-layer and the cnhaKeiiMitt-byer cm en^loy MP^ 
scalable coding. 

nOSi 6 and 7 show the DEC and INTP operatiais of decimation required for the 
•Huanawe iii(eriaoe^to4Dlerlaee fiatm of ^tatial scalabiU^. As shown in HG. 6. for 
» dnr hri a ti o n of fawertaced sjgnrt logenewiB a lower qyatlal lesrtutioo mtcrtoced signal, the 
first step invtdves at raterlace-to-piogiessive inteipolaion opeiBti<m. Assuming, for 
purposes of diis examine; that fa^wi rignal SlOn^j is 30 fhi^ 
can be ahernatdy viewed as 60 fidds/sec hiieilaced vkleo where eadi fieM contains 
the Itoes 1^ a frame. InteriaoB-toiiiogiesstve inietpoialor 6165 genermes popessive 

1 frames from the 60 MO/sce interlaced video signal iiqmt on line 6610 aich Oiat same 
number of lines as the ntfcriaoed fitame is outimt at 60 frames^sec on line 6105. 
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interiacc-n^progrcssive intcpolation. the «gnal is decimated in horizontal and vertical 

dec,mation6n5 which outputs the dedn««ed signal on Iine6l30 as SIGOUT„.Ifak^^ 
«solut»n mteriaced sigmJ with half Ae horizontal and vertical resolution as the original 
mpul s«nal is desired. d>en decimation fi«o„ of 2: 1 horiamtaUy and 4: 1 vertically may 
beemptoyed. 

FIG. 7 shows the INTP operation used in the ilhistrative interlace-to-interiace fom 
of spanal scabbiKty. Again, for purposes of dils example, lower spatial rtsoluUon 
signal SKHNo at 30fiB«es/sec is viewed as a 60 fields/sec signal. n« first step 
IS Identical to dial In decimation operation described above in reference to FIG. 6 In that It 
mvolvM mteriaceHoi,rogfessive interpolation operation on input signal SIGIN,, on line 
7150 wtach i««lts in a « ftanes/second progressive video output on Hne 7175. In the 
"«t step. dK progressive frames generated by even^ields ate available on line 7195 after 
passing through switch 7185 and ate vertically inten>olated by 1:2 and resampled by 
vertical interpolator and lii» selector 7235. Tbe progressive fhunes generated by odd-Helds 
ate available on line 7205 after passing through switch 7185 and are not resainpled in this 
step. Next, switch 7255 alternatively sekas the signals online7205and71 45. Inttie final 
Step. dK output of switch 7255 is fed via line 7145 to 1:2 horizontal inteipototar7155. 

It wai be hc^rfiil. at this point, to describe die interiace-to-progiessive inteipolatim 
operatioB employed in the DEC and IhTTP operations in more detail The interiace-to- 

pn)gressive interpolation openttion used in I»C and imp can be ^ite diffiwM dian dw 
for interlace-lo-interlace because, while die INTP operation is specified by die MPEG-2 

standanl. die DEC operatioo is outside of die standard and can du» be mote complicated. 

raos. 8 and 9 show ibt interiace-tO'inogressive interpdatiai operation inchided in 
QIC MPEO-2 standard. FIO. 8 shows hiteilace-lo-progressive iniarpoMon of fleldl fix., 
die odd-field), of an interlaced frame in a sequence of frames. Fig. 9 shows die interiace- 
to-progressive interpolation of &Id2 ( i*. die even-field) of an interlaced frame in a 
sequence of frames. 

HO- 8 allows die details of inicriacB-ii>{nogressive interpolation operation used m 
interiace-to-teteriace fonn of spatial scalatrili^. In HO. 8. lines A. C, E, O „. belong to die 
odd-fidd and lines B*. ly, P JP ... are generated by qiplicatitm of a filter. 

HO. 9 shows die princqdes bdund die weighted spado-temporal prediction for die 
pfOgtessive-io-ptDgi^sive and intetlace«>-piagressive fonns of spodal scd^ty in 
acccfdanec witti an aspect of die invention, bi HG. 9. lines B. D. F, H ... bdong to die 
even-field and lines A'. C, E*. H' are generated by the qipUcation of interlace-to- 
pragressive mteipolation filter. Tl» output of die interlace-to-progresave interpolation 
filler is cooqiosed of two conlriintiais. one ftmn die field being iteinteriaced, and die odier 
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ftom an opposite parity field wiihm the same ftame. Bod, contributions oe gmtd by 
apptynv weighting factois to samples of neighboring Kne, centered at the detoteriaced line 
to be genenled. as shc«Mn by the «mw8 in HGS. 8 and 9. TOs latering openiA^ 
"MntamsacomptomisebetweenietamiBgthevenfaUandthe 

An example of interlaceHo^nteriace scalability is a scalable codfaig scheme wheie 
b««^layer«Koder nW(FIG. l)encodesCCIR^U:2:0 resolution interlace 
enhanoement-teyer encoder 1 180 (FIG. I) encodes Hiyrv resolution interlaced fiames. 

Such interlace^c^interiace spatid scalaWHly can be advamagcously iised to ach^ 
interiaced-HDTV compatibility with standard digital TV. 

na 10 shows the principles behind the weighted spatio-tempoial prediction for the 
I«>gresri»wo.inleri«ce and iniefiace-to4Bteilace forn« of spatial sca^ 

widi an aspect of the invention. For progressive fonnat video rignals. na 10 sh«^ 
the opeiBtioo of d« spatio^emporal weighted predictioo applied to tfK spatial p^^ 
obtained by upsampling the locally-decoded pictnre fiom the base-layer and combining it 
with the modon<ompensaled temporal prediction to the enhancement-layer. IT* weight 

code W sunply represents weights w. where w b the spatial weight applied to all the Ihiee 
of each block. The spatio^poral weighted prediction is obtamed by weighting the 

spatial-prediction block by w and addmg to it die temporal prediction block weightBd ^ 
foctorofl-w. 

TaWes I and 2 show spatial weights, fior purpose* of this example only and not as a 
lirohation on tt» invention, for progressive-to-progressive and inleriace-to-progressive 

scatobiliiy. Besides the listed weights, porely teropotal prediction ft*, a spatial we^ 
0) may be used in each case. 

T»Me I. below, shows an example set of 2 wei^ codes for pcogressive-to- 
F^oSibssHk and itttetlaoe^o-pngresrive scalability. - 
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Tabte 2. below, diows an exan^ set of 4 weig^ codes for piogressive-to- 
progiessive and ntertaoe-tD-progressive scaldiOity 
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T^l«3«nd4sh(wspatial-wel8l«8.forpm^ 
h^n a« U» uwenOon. for pn^We^o-imerlace scalabiUty wh«e .he ba.e^ 
^ odd.»d even-fields respectively. Besides the Ibted weights. ^ .«„po«l 
prediction (te.. a spatial weight of (0.0)) may also be used. 

scalabdity, where die base-layer encodes odd-fields. 
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TaWe 4. below, shows an example set of 4 weight codes for pfogressive^o.inteI^ace 
scalabiUty, wtere die base-lay« encodes even-fields. 
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TEdte 5 and 6 shew spadalrweighls, for piin»se« of tfiisewmip^^ 
limitation on die mventioa. for progressive4o-interiace scalability where die base-layer 
eacodiM odd. and even^kls respectively. Besides the listed weights, purely lempoial 
Piedictioa (le.. a spatial-weigfit oS(OJJIi) may be used. 

TaWe 5. below, shows a set of 4 weight codes for piogressive^ointEilace 
scalaUliQr «toe die base-layer encodes odd-fidds. 
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Table ^. below, shows a set of 4 weight codes, fior pinposes of this example only 
and not as a Imitation on the invention, for progn«iw4o.iaiertaee scalabOiiy. when the 
base-toyer encjodes even-fields. 
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Tables b and 8 show spatial-weights. for purposes of this example only and not as a 
limitatioD on tk invention, for interiace-to-interlace scalability. Besides the listed weights; 
a purely taapdal predtctioo (iA, a spttial^ight of (OjO)) may be used. 

Tal* j. below, shows an examine set of 2 wdghl codte laertkMo-interiace 
scalaUlity. 



Wl 


w2 


1 


1 


1/2 


1/2 



Table 4 below. ihoTO an example set of 4 wci^^ 
scalabiliiy* ! M 
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FIG. n show, a diagram of the beseJayer encoder and an enhancontt-byer 
encoder .^in ite iih^mive enU«iin«M of HG. I. A 

MtKne 12100. SpttW 
a.d«cribed^«wl«mfcnin^ 

to base cnco.|er 12201. Base-teyer encoder 12201 uses the wrtl blown 
a«n«genw« ^ which, for genendity, codea I. B, and P pkmiie^ 
•wider. d« liiput famw in |«|w>^ 
and outputs die result on lines 12140 and 121S0. 

Molio* estimator 12170 examines Ae input frame on Hne 12150 and compues it 
With one or previously coded fhm«. If the input Is of type 1 or P 
|«vio«» Ihm* is used, if it is type B then two previously coded frames a« useA ^ 
otimator 12l!70 outputs modon veclois on One 12175 for use by motion conpensator 
mSOand oij Une 12185 for use by variable and fued lengdi encoder 12310. Modon 
coiBpensator j2180 utilizes motion vecton mv and pels from previously co^ 
compute (for ^ and B type frames) a modoo compensated prediction that is outpm on line 

12230 and pasjses to lines 12240 and 12250. For I ^ frames, motion compensator 12180 
outpute zero p^l values. 

Subtraitor I21tiO computes die difference between the input frame on line 12140 
and (for P amjl B types) die predictioa frame on line 12250. The resnh appean on Ime 
I22I5. b tiai|sfonned by transfocroer 12270 and quantized by quantiax^ 12290 into 
typlcaUy intedn- vahies. Onndzed transfmm ooeffidems pass on line 12300 to vnaUe 
andfixediengjbaK»dCT 12310and6nline 12305 toim 

favcrsej qnantiicr 12380 convem liie q^umdzed iransfonn «x)efrK^ 
range and pas^ die result via line 12390 to invene discrete cosine nansfamg 12400, 
which oiflpttts jpel pfediction enor vahies on line 124ia Adder 12420 adds die piedicdon 

error vahies oii Une 12410 to die predictioa vahies on line 12240 to form die coded b^ 
byerpels OB lijia 12430 and 12440. 

I a!|dP qipefiamies, switch 122Z5 passes dtt 
ioestiHctBn^stiiie 12200 via tine 12205. Snnnltaneously. die frame Uiat was in Bext-|wmiie 
store 12206 via Une 12195 to previoasi>icture store 12190. Fbr B type frames. 
swWi 12225 l^es no actfco, and die contents of picture stores 12190 and 12200 remain 
uiidianged. lli^ comnts of ptcnate stores 12190 and 1220Opass lo motion estimttor 12170 
ind motion ooijyensator 12180 via lines 12210 and 12220 for use as needed. 

The qu^tizer st^ size QS diat is used 1^ quantizer 12290 and inverse quantizer 
12380 Is oom^ted adqitively by qnamizalioo ad^ier 12360 <tepending aa die buffer 
Mtaiim iadical^ on Une 1235a Quntizer step size (25 p^ 
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12290 and 
variable and 
12310 

input on line 
output on 
12340 to 
intecpolator 
enhancement 
In 

resolution 
online 1248G. 
12480 that is 
prediction 
quffiitizi^ 
Quantizer 
quantization 
leceiyedon 
via tine 126l6 
encoder 12550. 
Une 124SS, 
ptevioQs 
enhancement-] 
motion vectoRi 
foruse l>y 
motion vector^ 
online 12700 

The 
line 12466 aw^ 
the ootpot of 
qpatio- ie nipo ii^ 
the motion 
temporal 
the thtid input 
weighing 
scalaWUty, and 
setctf weigfais 



enlmncement-l 



video 



enor 
12230 
step 



line 



■aid 



IS 

line I2375toinveiseqiimtizer 1238a <>ianeiar step «^ 
Ji«d leiigih encoder I23I0 via line 12365. VaiiaMe 

quantized transfomi wfBciems ini»^ 
12185 and qnamittr step si» QS input on line 12365 into a bit-sneam that is 
12320 into a bttlfc 12330 for tempoiaiy storage unta it passes via line 
multiplexer 123«. Tl» coded base layer frames pass via line 12440 to 
2450, as described ^>ove. wfaeie diey an apsmpled and passed to tfw 
layer encoder 12201 via line 12460. 

t-toyer encoder 12202. ftarae organizer 12470 leoideis die high 
fifames to match tho Older of the base^ and outputs reordewl franes 
Subtracter 12490 computes the diffeieoee between dK input picture on Kne 
to be coded and the spati(Heniporal piedictioa picture on line 12460. TTie 
is output OB line 1250Qi. transfonned by transfomier 12510. quantized by 

and passed via Hne 12S40 to vaiiaWe and fixed length encoder 12550. 
size QSg used by enhancement-hyer encoder 12201 is confuted by 
ajtMer 12600 depending on die an indication of the fiiUness of buffer 12570 

12590. Qiamizer step riae 0% passes via line 12605 to quantizm2S30. 
to inverse quantizer 12740 and on line 12615 to variable and fixed lengdi 
Motion Estimatar 12640 examines the enhancement-layer mpnt frame on 
(iepeoding on the picuue Qipe being coded, compaes it with either the 

J oihancemeiifrlayer on line 12630 or widi two previous decoded 
layer &Bmes on lines 12630 and 1268a Modon Estimator 12640 outputs 

rnvg on line 12650 for use by motjon con5)ensatot 12655 and on line 12645 
and fixed length encoder 12550. Motion Compensator 12655 utilizes 
mve to oonvniB a nodon conyensated ten^tml prediction diat is outptt 
«d passes to wcigfater 12710. 

KspondingsiMtially interpolated base-layer decoded frame is available on 
is input to weighter 12710 on line 12690. TTie spntial prediction frame at 
he Inse-bjcr inieipoiaior 12450 is also qiplied to « input line 126^ 
' weighting analyzer 126S5< Hie ten^oral prediction frame at d» on^ of 

ccmpensator 12655 is also applied to anodier iiq>ut Hne 12675 of spatio- 

^g analyzer 12685. The ii^ut frame fiom frame organizer 12470 is fed to 
line I27QS <tf spuSo^mponl weigliting analyzer 12685. Spatio-temporal 
1268S find selects a weigjiting taUe <t«yM«««rg cm die ^pe of «p«Tm 
next computes an index to this prestored taUe indicating die best weight or 
to be used. Hie operation can be done once or more per pcture. TypicaUy, 
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Wetgliter 12^10 computes 
and 12700 
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1 2780» respectively. 



The 
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coeffidents 
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I spatio-temporal 



IS 

enhancemenH[^ 
layer decoder 
' gen^relhyy 
from the 
line 13320 to 
decoder 13310 
Quantizer step 
13360 and 
and peb from 
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an MPEG-like codiqg environment, spatio-teropoml weights are adapted 
by macroblock basb. Tlie index to selected wetghis for a roaaoUock 
12695 and b fed to weighter 12710. Thb index also appeals on line 12725 
as part <^ the bit-stream in variable and fixed length encoder 12550. 
a weighted aver^ of die two predictions input on lines 12690 
outputs the result on lines 12720 and 12730 to subtiactor 12490 and adder 



^ly decoded enhancement-layer video, which b needed for motion 
of Ae next enhanoemem-laycr frame* b calculated in the same way as in Oie 

for a few diffacnces. Specificany, the quantize 
to full range by inverse quantizer 1 2740, converted to predictioo enor pel 
transfom 12760, added to die motion coii^iensated predictk^ 
' to die next-fiame store 12620 whose contents can be simultaneously 
ptevious-ftame store 12660. If the next fiame b a P picture, contents of 
store are needed, if it b a B picture contents of both frame stores are 
ijotioo estimation. Variable encoder 12550 encodes quantized transform 
on line 12540. quantizer step siaes QSg input on line 12615, motion 
line 12645 and hxiex of weights on line 12725 into a bit-stream that b 
12560. Thb bit-stream on line 12560 dien passes to buffer 12570 for 
unti} it passes via line 12580 to systems multiplexer 12345. 

of thb exanqple a sim|te encoder is used to ilhistiate the base and 
layer encoders described above. However, it m^ desirable that die base- 
be an MPEO-1 or H.261 encoder or an MPBG-2 mamijiofifc enc^ 
layer encoder b assumed to be an MPEG.2 next-|Hofile spatial scalainli^ 
" b similar to an MPeG-2 mam-profile encoder excqit for wcighter 12710 
analyzer 12685, which are dbcussed in detail below, 
shows a twivlaytr decoder consbting of base-byer decoder 13001 and 
decoder 13002 conesponding to the coding system of FIG. 1. Base- 
13001 uses the weU known MPEG-1 pic^ 

of I, B. and P jpicnnes. The leoeived bit-stieam on Ime 13340 passes 
demultiplexer to buffer 13330 for teii ip oi ai y storage until it passes via 
jhe variable and fixed length decoder 13310. The variable and fixed length 
decodes quantized iransfoim coefficients and outputs diem on line 13300. 
size QSp b ou^ on line 13370. Motion vectofs mvp are output on Ih^ 
Motion compensator 13180 utilizes nH>ti<m vectors mv|^ ^n line 13175 
previously decoded frames on line 13210 for P pictures and previously 
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decoded 
prediction it 
OQ^utszero 



frames 



Quantizer 



» invirse 



quanizer 



bicki 



I ecDr 



base- layer pels < 
Simul! aneously. 



picture 

and : 
aad 



nnu 



13370 to 
inveise 
coefficients 
13400, which 
prediction 
decoded 
13435 passes 
13200. 

pievious-picti|e 
contents of 
stores 13190 
lines 13210 
reorders the 
line 13125 in 
line 13440 to 
toenhanoemenlt 
Hie 

buffer 
and fixed 
quantized 
aie output on 
index of 
from line 
quantized 
line 13750 to 
IJ77a 

Motion 
Uite 13650 
and 1368010 
passes to 
13450 and qppl^ed 
computes a 



13570 vSa line 
lei^^ 
transform ( 

line] 



^ and pels 



weig liter 



on lines 1 321 0 and 1 3220 for B pictures to compute motion compensated 
-s output on line 13240. For I type frames, motion compensator 13180 
values. 

step QSd passes from variable and fwed length decoder 13310 via line 
quantizer 13380. Quantized transform coefficients pass on line 13300 to 
- 13380. Inveise quantizer 13380 converts the quantized transform 
to fuUrai^ and passes the rcsuk via line 13390 to Inverse transfoimer 
outputs pel prediction error values on line 13410. Adder 13420 adds the 
values on line 13410 to the prediction values on line 13240 to form the 

' <»Knes 13430. 13140 and 13440. P6r I and? type fian« 
decoded pck input on line 13430 via line 13205 to the ncxi^ncture 

the fran» m next-picture store 13200 passes via line 13195 to 
store 13190. For B type frames, switch 13435 takes no action, and the 
stoies 13190 and 13200 remain unchai^. Hie contents of picture 
1 3200 pass tt> moUon estimator 13170 and motion conqiensator 1 3180 via 
13220 for use as needed as described herein. Frame CTganizcr 13130 
^-laycr decoded ouqput frames on Kiie 13140 in preparation for display on 
aie manner weH known in the art Hie decoded base*layer frames pass via 
itcrpolator 13450, as described above, where they are upsan^led and passed 
^ layer decoder 13002 via line 13460. 
enhancement-layer bit-stream passes from systems demultiplexer 13005 to 

13580 Ibr ten^oraiy storage until it passes via line 13560 to varia^ 
decoder 13550. Variable and fixed lei^ decoder 13550 decodes 
coefRcients and outputs them on line 13540, quantizer step size QSi^ 
13610. motion vectors mvpg are output on lines 13645 and 13650 and an 
are output on lines 13725 and 13695. Quantizer step size QSpg passes 
to mverse quantizer 13740. The faiverse quantizer 13740 convMs the 
coefficients on line 13540 back to full range and passes the result via 
inverse transform 13760. which ouqwu pel prediction error values on line 



13610 
t transform < 



conq)ensator 13655 utilizes enhancemem-layer motion vectoiB mvpg on 
" from the previously decoded erfianccmem layer frames 
compute a moticm compensated prediction that is ou^jut on line 13700 and 
13710. Hie decoded base-layer frame is iqisampled in interpolator 
via line 13460 to die other input to die wei^iter 13710. Wdglner 13710 
averse of the two predictions input on lines 13460 and 13700 and 
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r»alt on line 13720 to adder I37«0. The weightbg used in computing the 
he same as thai used during the encoding process. TTie weights a^ 
I ^710 by using index of weights available at line 1 369S to look up values frnn 
^tput of adder 13780 is avaiU>le on lines 13480 and 13790, decoded fhme 
If not a B [Ncuire, the output is passed through switch 13810 to line 13815 
the next picmit store 13620 aid its coMents are sinwltaneously o^ied to 
store 13660. The contents of picture stores 13660 and mw are used for 
prediction of subsequent frames. Frame reorganizer 13470 reofders 
video frames on line r3480 to matdi the oider of the base-1^ and 
on line 13135 for display, 
pijuposes of this example a simple decoder is used to iUustrate the base and 
descidied above. However, it may be desiiabie diat the base- 
be an MPEQ-l or m 11261 decoder or an MPEG-2 main-profne decoder, 
t-layer decoder is assunvd to be an MPE0.2 next-pnrfile spatial 
which is very much tike the MPK}-2 main-profile decoder except for 
^ which b discussed in detail below. 

1 4 shows details of die spatio-tempoial analyzer en^yedm FIG. l2.Spetio* 
14000 takes die ^ial prediction signal obtained by intenx>lation of die 
on line 14690 and die enhancement-layer temporal pitdiction signal on 
signal on line 14690 also appears on Ime 14880 and the signal on line 
on line 14890 and tfus form two inputs to qwtio-lenvoral weighter 
mpnt is weight(s) WO on line 14910. The fust entry in die weight look-up 
hiving contents Am, are available on line 14905. This process is repeated for 
wei^teI4^| which takes signal on lines 1 4690 and 14700 at reqjcctive 
and 1489K widi weight Wi on Ime 1491 K die next entry in die wdght 
14870 whose contents are available cm line 14905. This process is similarly 
q»ado-ttn9cral weighters^ wbm die number of weig|«ers depmls on 
availabte to choose from in die weight tables. The spatio-tempcxal 
iniage^iloda are now available on lines 14930, 14931, ^ 14934 and 
in diffiBcenGers 14950* 14951.... 14954 from oiginal hnage^UodLS 
Ines 1494p, 14941, ... 14944, die resulting prediction mxK blocks are 
l^60» ]4961»». 14964 and form an input for oomputadon of a single 
per hnage-bloGk per spado-temporal predkdon. Each distortion 
hi 14970, 14971,..]4974 is eidier sum of squares or sum of absolute 
Mitput on lines l^W, 14981.., 14984 and fmns inputs to die conq)arator 
all de values and ou^ an hidex on Une 1 4695 conesponding lo 
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per image4>locL If MPBO-1 type coding is employed, an hnage>block is 
macroWoct In diffcrem contexts, an im^ 



look-up table 14870 contains separate weight tables for die various 
of spatial scabibiUty (i^, piogiBssive^oiwogressive, interbice^ 
toixogressive and interiace-io^inierlacc) and, dependii^ on the form of 
chosen, a conespuuding weight table is selected, 
shows the details of the spatio-tempoial weighter employed m RGS. 12, 
!j»tio-temporal weighler 15000 takes spatial predictioo signal obtained by 
-the base.layer signal on line 15690 and the enhancement-layer temporal 
online 15700. An index input on line 15695 is ^Ued to form an address 
laWe 15870, and the lesuhh^ weight(s) on line 15825 are applied to a 
having a spatial prediction signal on line 15690 as its odier Input Also, 
are an>lied on line 15845 forming an input to the diffeiencer 15865. The 
forced to value of M.' and the resulUng con^ementary wcight($) signal 
Ihie 15835, which in turn forms an ii^t to the multiplier 15830. The 
signal on line 15700 forms the odier n^t to the muldplier 15830. The 
multipliers 15820 and 15830 are appear on lines 15840 and 15850 md form an 
15860. The ouqiut of adder 15860 is the spatio-ten^oral weighted 
and appears on line 15875. 
abm^described invention provides a tecfaniqne for deriving video images of 
finora a single video source. It will be understood that the particular 
are only ilhistrative of die principles of the present invention, and that 
na couU be made by those skilled in die act widiout departing from the 
scop< I of the piesem inventioii, which is liinited only by the du^ 
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